Multi-touch technology has become a standard modus of interaction with new touch screens. Such screens are becoming more pervasive, making their way into mobile technology and allowing for evolving forms of interaction. Such touch screen technology has enabled a shift in computing into a new brand of devices, including tablet computers, mobile devices and other touch screen supported systems. The problems with various single and multi-touch systems are many. Smudging is the most obvious one, in which the screen becomes unclear from regular usage. Hygiene is another issue in which the screen, especially if used by more than one person, may become unhygienic. Reliability and dependability of the screen are other considerations, along with the price associated with integrating a touch screen into a tablet device. Responsiveness that is associated with multi-touch gestures is another issue, since there exists a significant lag in the operation of the system itself.
A cheaper and more effective solution is to utilize optical sensors embedded in the bezel of the screen at various locations. Systems with sensors arrangements in the bezel are already available on tablets and other systems, and some are being made available in the near future. Optical-based interaction, if successful, is cheaper to install than a touch screen, and provides for a better and more reliable viewing quality. To enable such an optical based interface, depth has to be constructed in an area close to the screen's surface, i.e. a three dimensional representation of any objects positioned close to the screen's surface must be constructed so that movement relative to the screen may be determined. Once depth is calculated, replacing multi-touch gestural events with near-touch events that are emulating the same behavior becomes trivial.
There are no methods available today to reconstruct depth at such a close distance for real-time applications. Some work exists in the literature on spherical stereo, such as that presented in Li, Shigang. “Binocular Spherical Stereo” IEEE Transactions on Intelligent Transportation Systems (IEEE) 9, no. 4 (December 2008): 589-600. however such methods are computationally expensive and lack robustness for them to be applicable to a tablet computer. In accordance with embodiments of the present invention, a different stereo approach, presented in U.S. patent application Ser. No. 13/025,038, filed Feb. 10, 2011 by El Dokor et al., titled “Method and Apparatus for Segmentation of an Image”, the contents thereof being incorporated herein by reference, may be employed. A similar dewarping algorithm to that used by Li (noted above) may also be employed as a preprocessing step to the algorithm. A detailed description of this stereo algorithm has been presented in, for example, the above referenced '038 patent application. Building on the similar concepts of disparity computation with stereo, in accordance with the present invention, a novel approach is utilized for the implementation of near-touch systems.
There are other methods that are known to one of ordinary skill in the art for performing depth computation other than computational stereo. One such method utilizes an active light source with a structured pattern (these methods are sometimes referred to as active stereo). A pattern may be projected onto the field-of-view (FOV) from a light source close to the imager, and then the distance between the individual patterns is calculated. Some examples of this approach include Microsoft's Kinect™ Xbox™ peripheral, and PrimeSense's ASIC+Prime Sensor, as described in Meir Machline, Yoel Arieli Alexander Shpunt and Barak Freedman, “Depth Mapping Using Projected Patterns”, May 12, 2010, where various spots of light are utilized for the calculation of a depth map. In addition to the fact that the light source is usually in the infrared range, and requires a significant amount of energy, active stereo suffers from a number of practical and computational drawbacks. For instance, such methods are usually utilized to address depth computation at extremely large distances (relative to the screen, in the context of a few inches), forcing the light source to be very power-consuming and bulky. The data from the depth images have to be evaluated over a number of frames, making the data sluggish and introducing a significant amount of motion blur and computational artifacts that become apparent in performance. Such methods are inappropriate for near-touch interaction.
Therefore, it would be desirable to present a near touch system that overcomes the drawbacks of the prior art.